Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write new blocks and states to the database atomically #1285

Merged
merged 9 commits into from
Jul 1, 2020

Conversation

adaszko
Copy link
Contributor

@adaszko adaszko commented Jun 23, 2020

(Mostly) fixes #692

@michaelsproul michaelsproul added the ready-for-review The code is ready for review label Jun 24, 2020
@adaszko adaszko requested a review from paulhauner June 24, 2020 15:24
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Can't wait to merge this so we have real atomic DB operations!

I've left one major(ish) comment and a few minor ones

beacon_node/store/src/state_batch.rs Show resolved Hide resolved
beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
store.store_cold_state(&state_root, &state)?;
let mut ops: Vec<KeyValueStoreOp> = Vec::new();
store.store_cold_state(&state_root, &state, &mut ops)?;
store.cold_db.do_atomically(ops)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This strikes me as odd when maybe the whole migration should be 2 DB transactions (copy to freezer, delete from hot), but I'm happy to postpone thinking about that until later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I overlooked this one. Happy to correct this in a separate PR.

) -> Result<(), Error> {
let total_timer = metrics::start_timer(&metrics::BEACON_STATE_WRITE_TIMES);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This timer should probably stay for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be meaningless. The PR makes it so that the actual db write is executed at some other place in the code, during the call to do_atomically().

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, of course! Lets delete BEACON_STATE_WRITE_TIMES from beacon_node/store/src/metrics.rs then, as it's now unused. I've also opened #1307 to track the state of the store metrics

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done ✅

beacon_node/store/src/lib.rs Outdated Show resolved Hide resolved
beacon_node/store/src/lib.rs Outdated Show resolved Hide resolved
beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to merge once the conflict is resolved!

@michaelsproul michaelsproul added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Jun 29, 2020
@adaszko adaszko force-pushed the master branch 2 times, most recently from ccd4307 to e341c55 Compare June 29, 2020 10:11
@adaszko adaszko added ready-to-squerge and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Jun 29, 2020
@michaelsproul
Copy link
Member

@paulhauner I've just tested this on Altona and it's performing well 💯 If you're OK to merge it I'll go ahead and merge tomorrow

Copy link
Member

@paulhauner paulhauner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A great feature implemented elegantly! Love it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use atomic operations in database to harden against failures
3 participants